Stochastic Optimal Control for Small Noise Intensities: The Discrete-Time Case

This paper deals with Markov Decision Processes (MDPs) on Borel spaces with an infinite horizon and a discounted total cost. It will be considered a stochastic optimal control problem which arises by perturbing the transition law of a deterministic control problem, through an additive random noise term with coefficient epsilon. In the paper, we will analyze the behavior of the optimal solution (optimal value function and optimal policy) of the stochastic system, when the coefficient epsilon goes to zero. Specifically, conditions given in the paper guarantee the uniform on compact sets convergence of both the optimal value function and the optimal policy of the stochastic system to the optimal value function and the optimal policy of the deterministic one, when epsilon goes to zero, respectively. Finally, two examples which illustrate the developed theory are presented. Key–Words: Stochastic Optimization, Markov Decision Process, Dynamic Programming, Total Discounted Cost, Deterministic Approximation, Inventory/Production System